: In this paper, the operation scheduling of Microgrids (MGs), including Distributed Energy Resources (DERs) and Energy Storage Systems (ESSs), is proposed using a Deep Reinforcement Learning (DRL) based approach. Due to the dynamic characteristic of the problem, it firstly is formulated as a Markov Decision Process (MDP). Next, Deep Deterministic Policy Gradient (DDPG) algorithm is presented to minimize total operational costs by learning the optimal strategy for operation scheduling of MG systems. This model-free algorithm deploys an actor-critic architecture which can not only model the continuous state and action spaces properly but also overcome the curse of dimensionality. In order to evaluate the efficiency of the proposed algorithm, the results were compared with the analytical method and a Q-based learning algorithm which demonstrates the capability of the DDPG method from the aspects of convergence, running time, and total costs.

This paper proposes Improved Model-Based Deep Deterministic Policy Gradient (IMB-DDPG), a novel reinforcement learning algorithm designed to overcome three critical challenges in industrial DRL applications: (1) poor sample efficiency requiring excessive real-world trials, (2) safety risks from unstable policies during training, and (3) difficulty scaling to high-dimensional continuous control spaces. Building on DDPG's strengths for continuous control, IMB-DDPG introduces four key innovations: (i) a Virtual Environment (VE) for data-efficient learning, (ii) a Simulation Rate (SR) mechanism adapting model reliance dynamically, (iii) a Simulated Experience Buffer (SEB) preventing divergence, and (iv) a Performance Threshold (PT) for fail-safe operation. Evaluated on Cart-Pole benchmark via OpenAI Gym python library, IMB-DDPG demonstrates faster convergence than standard DDPG while maintaining performance degradation under sensor malfunctions or communication losses. These improvements derive from the algorithm's unique ability to simultaneously leverage real-world data and model-generated experiences, reducing physical trial costs while ensuring operational safety. The results establish IMB-DDPG as a practical solution for industrial control systems where reliability and data efficiency are paramount, particularly in applications like chemical process control and precision robotics that demand stable operation amid sensor/communication failures.

The hybrid electric train which operates without overhead wires or traditional power sources relies on hydrogen fuel cells and batteries for power. These fuel cell-based hybrid electric trains (FCHETs) are more efficient than those powered by diesel or electricity because they do not produce any tailpipe emissions making them an eco-friendly mode of transport. The target of this paper is to propose low-budget FCHETs that prioritize energy efficiency to reduce operating costs and minimize their impact on the environment. To this end, an energy management strategy [EMS] has been developed that optimizes the distribution of energy to reduce the amount of hydrogen required to power the train. The EMS achieves this by balancing battery charging and discharging. To enhance the performance of the EMS, proposes to use of a deep reinforcement learning (DRL) algorithm specifically the deep deterministic policy gradient (DDPG) combined with transfer learning (TL) which can improve the system's efficiency when driving cycles are changed. DRL-based strategies are commonly used in energy management and they suffer from unstable convergence, slow learning speed, and insufficient constraint capability. To address these limitations, an action masking technique to stop the DDPG-based approach from producing incorrect actions that go against the system's physical limits and prevent them from being generated is proposed. The DDPG+TL agent consumes up to 3. 9% less energy than conventional rule-based EMS while maintaining the battery's charge level within a predetermined range. The results show that DDPG+TL can sustain battery charge at minimal hydrogen consumption with minimal training time for the agent.

This paper presents an improved framework for deep reinforcement learning algorithms integrating online system identification, based on the Dyna-Q architecture. The proposed framework is designed to tackle the challenges of both Multi Input Multi Output (MIMO) and Multi Input Single Output (MISO) systems in complex, industry relevant environments, thereby significantly enhancing adaptability and reliability in industrial control systems. It should be noted that in the suggested novel framework, the system identification and model control processes run in parallel with the control process, ensuring a reliable backup in case of faults or disruptions. To verify the efficiency of the aforementioned approach, comparative evaluations in the presence of three of the most common deep reinforcement learning algorithms, i.e. Deep Q Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3), are conducted on industry-relevant environments simulations available in OpenAI Gym, including the Cart Pole, Pendulum, and Bipedal Walker, each chosen to reflect specific aspects of the novel framework. Results demonstrate that the proposed method for leveraging both real and simulated experiences in this framework improves sample efficiency, stability, and robustness.

Facial expression recognition (FER), which is one of the basic ways of interacting with machines, has attracted much attention in the recent years. In this paper, a novel FER system based on a deep convolutional neural network (DCNN) is presented. Motivated by the powerful ability of DCNN in order to learn the features and image classification, the goal of this research work is to design a compatible and discriminative input for pre-trained AlexNet-DCNN. The proposed method consists of 4 steps. First, extracting three channels of the image including the original gray-level image in addition to the horizontal and vertical gradients of the image similar to the red, green, and blue color channels of an RGB image as the DCNN input. Secondly, data augmentation including scale, rotation, width shift, height shift, zoom, horizontal flip, and vertical flip of the images are prepared in addition to the original images for training DCNN. Then the AlexNet-DCNN model is applied in order to learn the high-level features corresponding to different emotion classes. Finally, transfer learning is implemented on the proposed model, and the presented model is fine-tuned on the target datasets. The average recognition accuracies of 92. 41% and 93. 66% are achieved for the JAFFE and CK+ datasets, respectively. The experimental results on two benchmark emotional datasets show a promising performance of the proposed model that can improve the performance of the current FER systems.

Aluminum alloys are using widely duo to high strength-to-density ratio in the industries of automotive, shipbuilding and aerospace as a substitution of steel sheets. .To increases the formability of aluminum alloys in deep drawing process and due to formability problems of these alloys in room temperature using of warm deep drawing process is necessary. According to recent researches, warm deep drawing in gradient condition has better results as isothermal case. In this paper the process parameters in production of cylindrical parts from aluminum alloys 5083 sheet with 2mm thickness is investigated. For this purpose, gradient warm deep drawing in temperatures of ambient (25˚C), 80˚C, 150˚C, 180˚C, 250˚C, 350˚C , 450˚C and 550˚C have been used. The blank in flange region is heated by die heating and the blank center to increases the strength of the region which contact with punch corner radius is cooled by water circulating punch. The results show that increasing the temperature of the blank in flange region and also cooling of blank center lead to improve the limit drawing ratio. In forming temperature of 550˚C and ram speed of 378 mm/min and lubrication by graphite powder can reach to the limit drawing ratio equal to 2.83.  

Background: T1 thermometry is considered a straight method for the safety monitoring of patients with deep brain stimulation (DBS) electrodes against radiofrequency-induced heating during Magnetic Resonance Imaging (MRI), requiring different sequences and methods.Objective: This study aimed to compare two T1 thermometry methods and two low specific absorption rate (SAR) imaging sequences in terms of the output image quality.Material and Methods: In this experimental study, a gel phantom was prepared, resembling the brain tissue properties with a copper wire inside. Two types of rapid gradient echo sequences, namely radiofrequency-spoiled and balanced steady-state free precession (bSSFP) sequences, were used. T1 thermometry was performed by either T1-weighted images with a high SAR sequence to increase heating around the wire or T1 mapping methods.Results: The balanced steady-state free precession (bSSFP) sequence provided higher image quality in terms of spatial resolution (1×1×1.5 mm3 compared with 1×1×3 mm3) at a shorter acquisition time. The susceptibility artifact was also less pronounced for the bSSFP sequence compared with the radiofrequency-spoiled sequence. A temperature increase, of up to 8 ℃, was estimated using a high SAR sequence. The estimated change in temperature was reduced when using the T1 mapping method. Conclusion: Heating induced during MRI of implanted electrodes could be estimated using high-resolution T1 maps obtained from inversion recovery bSSFP sequence. Such a method gives a direct estimation of heating during the imaging sequence, which is highly desirable for safe MRI of DBS patients.

This paper presents and compares two different methods using in the forecasting of wind power turbine (WPT) outputs. These two forecasting methods, which utilize different types of input to forecast the output of WPT, are the Meteorology Forecasting Method (MFM) and the Observational Forecasting Method (OFM). The MFM determines the unit output from the forecasted wind speed at the WPT installation site, using the input from a composite data set created from the original annual-hourly weather data. Three different techniques can be used in MFM to forecast the wind speed, and the best result is selected for conversion calculation of the output of WPT. OFM, however, forecasts the unit output based on five observed annual-hourly data obtained from the operation of target WPT. Two different techniques can be used in the OFM simulation. The results from these techniques for each method are compared and the best one will be used for the final forecast of the WPT outputs. This paper presents and compares the forecasting results of WPT output obtained from MFM and OFM. Furthermore, in order to increase the result precision and decrease the forecast error, a new composite data system is also developed and proposed.The methodologies proposed in this paper will be very useful for designers, planners and operators of the wind power turbines